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DEVELOPING AN INSTRUMENT TO ASSESS 
HIGHER EDUCATION LEADERSHIP 

Introduction 

There is an irony to higher education. On one hand, it is big business— $280 billion in 
2000-2001, 15 million students, 1.8 million employees. It has made tremendous progress in 
shifting from an industrial to technological base, it fills a demanding workplace with bright and 
better trained students, and it continues to hold its stead as the key to economic prosperity and 
social mobility. On the other hand, it is regarded as “ill-suited” and “ill-adapted” to the very 
technology, skills and knowledge that it purports to advance. Among the reasons is the failure of 
its leadership— leaders are criticized for not knowing how to lead (Bensimon, Neumann & 
Bimbaum, 1989; O’Brien, 1998; Rhodes, 2001; U.S. Census, 2002). 

The development of leaders of institutions of higher education (IHEs) has been a source 
of concern in view of the fact that so many come to their positions with less than adequate 
preparation (McDade, 1987). Because administrators are either “brought up” from the faculty 
ranks or “brought in” from disciplines outside the academy (business, law, ministry), their 
preparation has come in the form of on-the-job training or post-appointment professional 
development programs (Allen & Cherrey, 2000; McDade, 1987). 

Originating discipline notwithstanding, leadership is the common thread in higher 
education administration, so it is important to ensure that developmental programs target 
leadership for that specific context. Unfortunately, leadership is often treated as a general topic 
across disciplines rather than being developed for the institution in which it is practiced (Bass, 
1990). Traditional leadership assessment instruments, then, overlook the specific context of 
higher education, providing little systematic knowledge for higher education administrators 
about behaviors, leadership styles, and effectiveness in IHEs (McDade, 1994; Williams, 2001). 
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Indeed, no instruments exist that specifically assess higher education leadership. 

This paper describes the development of an instrument to assess the construct of higher 
education leadership, reports the outcomes of that study, and relates some of its findings to the 
professional development of higher education’s leaders (Montez, 2002). 

The Higher Education Environment 

Shared Governance 

The unique nature of American higher education sets it apart from the rest of the world— 
the structure of its governance is more horizontal than hierarchical, and the institution’s 
leadership is expanded to include faculty, administrative, regent and, to a lesser degree, students 
(Bensimon, Neumann & Bimbaum, 1989; Fisher & Koch, 1996). In this environment the “pull 
and tug between faculty and administrators” effects reform (O’Brien, 1998, p. xiii), and protects 
the autonomy and academic freedom of its faculty (Vroom, 1983). When it is administered 
effectively, the “campus governance machinery” invokes the collective intelligence of the 
campus community to common advantage (Fisher & Koch, 1996, p. 144). When it is not, that 
tension between factions undermines the very nature of administration in IHEs; conflicts arise 
and result in unclear institutional goals and a perceived lack of management (Bensimon et al., 
1989). 

Thus, the organizational work of academic leaders requires dynamic, collaborative, cross- 
functional teamwork, instead of emphasis on a hierarchy of authority (Williams, 2001). In other 
words, if only one aspect of authority is developed, then the entire administrative effect will be 
compromised (Green, 1988). In higher education, leadership must be regarded as the effort of 
specialized but synergistic factions. 



External Forces 
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The environment of higher education is also continually affected by forces outside of it 
that shape its programs and the work of its leaders. Some of these forces are (a) the changing 
demographics of higher education’s students, (b) demands for greater fiscal accountability, (c) a 
fast-changing technology, and (d) growing competition from in-house or proprietary institutions. 

The demographic makeup of students entering higher education reflects growing numbers 
of women, ethnic minorities and "older" students in search of a more practically based education 
(Keller, 1983; Murdock & Hoque, 1999). The aging of the populace and the increase in number 
and proportion of diverse ethnic groups in the U.S. require astute leadership in higher education 
that is aggressive and active in recruitment, retention, remediation, and fund-raising (Murdock & 
Hoque, 1999). 

Fiscal challenges from federal and state governments abound as colleges and universities 
face decreased resources and increased expectations of accountability (Keller, 1983). Prisons, 
human rights commissions, health and welfare programs, and Medicaid all vie with IHEs for 
precious funding (California Higher Education Policy Center, 1994). The public's concern about 
the increased cost of tuition thwarts higher education leaders’ attempts to either justify such 
increases or land alternative sources of funding (Keller, 1983). This concern is also echoed in 
demands that higher education’s funding come from those (students, parents, employers) who 
directly benefit from it (“A Little Learning,” 1997). These external pressures are forcing 
universities to cut costs wherever possible while simultaneously implementing new 
accountability measures and answering the public skeptics (Marcus, 1997). 

The networked work world today seeks stronger ties between educational and practice 
systems (Curry & Wergin, 1993). Technology, though welcomed and endemic to higher 
education, is another force that weighs against IHEs. IHEs face the expectation that their 
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curricula mirror the reality of the world outside it; conflicts arise when the traditional modes of 
instruction are supplanted by an information technology that provides expanded learning 
possibilities (Privateer, 1999). The “increased connectivity” (Allen & Cherrey, 2000, p. 1) and 
integrated networks in higher education today has changed the delivery of education and students 
are more technologically based than liberally educated (Levine, 2000). The lines drawn by the 
faculty between skill-based training and theory-based education also often undermine leaders’ 
efforts in the process of instituting new and innovative curricula, and leaders of IHEs must 
carefully balance the rapid technologic change with the faculty’s resistance to it (Wolverton, 
Gmelch, Montez &Nies, 2001). 

Traditional higher education is now competing with different forms of post-secondary job 
preparation. More than $60 billion are spent on education and training in the workplace; these 
in-house mini-universities are corporate America’s response to its dissatisfaction with the skills 
and abilities of college graduates (Thompson, 2000). In addition, proprietary (for-profit) 
schools generally provide short-term occupational training to a high percentage of women, 
minorities, and low-income students (Goodwin, 1991). Others, such as the University of 
Phoenix, are accredited and grant degrees (bachelor’s, master’s, and Ed.D.s). Public officials 
warn that “knowledge industries” that can deliver information to untrained workers will replace 
university teaching (Eamon, 1999, p. 200). IHE leaders are stymied as they try to advance the 
merits of a full-blown education to a population that once could cannot afford it, but which now 
has viable alternatives. 

Leadership for the Higher Education Environment 

Given the complexity of higher education, an effective leader must align the internal 
environment and its external forces (internal and external) and emphasize the interdependence 
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between them. Administrative skills and background knowledge inform the bulk of the 
requirements for these leaders. Technical skills in budget and finance and computers, as well as 
knowledge of laws and legal issues affecting higher education, have been identified as areas in 
which leaders need training (Townsend & Bassoppo-Moyo, 1996). Curricular expertise and 
interpersonal relations are other areas of administrative knowledge that they must possess 
(Townsend & Bassoppo-Moyo, 1996). 

But technical and administrative skills alone do not make for effective leadership. 
Attitudes, behaviors, and beliefs form another perspective from which to examine effective 
leaders (Bogue, 1994; Townsend & Bassoppo-Moyo, 1996). For example, Daughdrill (1988) 
described passion, vision, stewardship, courage, and an ability to implement different talents at 
different times as the essential attributes of leaders who can effectively navigate higher 
education’s environment. Leaders are also charged with having to efficiently and effectively 
resolve the tensions that arise in the process of adapting the learning community and business 
management techniques (Kauffrnan, 1990). The challenges and opportunities in the 
administrative and leadership ranks of colleges and universities create expectations of leaders 
who can speak not only to curricular offerings but as well to campus climate, civility, behavior, 
and lack of diversity (Kauffrnan, 1990). 

However, little more exists in the higher education literature that describes appropriate 
behaviors and attributes of persons for leading in this unique environment. To complement the 
information from these limited sources, a limited email survey was next conducted to compile a 
“list of critical behaviors” and attributes of current higher education leaders (Crocker & Algina, 
1986). This list emerged along a behavioral dichotomy that described leaders’ behaviors (a) 
demonstrated toward others and (b) those they possessed. 
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Following the limited survey, the leadership literature was revisited to further explicate 
the construct of higher education leadership by identifying theoretical bases for these behavioral 
categories. Several better-known, conventional leadership theories were examined. These 
included trait (e.g., Gabon, 1869), personal-situational (e.g., Blake & Mouton, 1964; Hersey & 
Blanchard, 1969), perceptual-cognitive (e.g.. Green & Mitchell, 1979; Vroom & Yetton, 1974), 
and the transformational/transactional paradigms (e.g,. Bass, 1990; Bums, 1978; Tichy & 
Devanna, 1986). 

These theories emphasize the attributes and behaviors of individual leaders and presume 
the strength and will (also couched in terms of “influence” and/or “charisma”) of leaders and 
their effect on followers. While these theories explain the interaction between a leader and a 
follower, they fail to account for the context in which such relationship occurs (Yukl, 1999). 
They also presuppose a hierarchy, where power emanates from leader to follower, as a condition 
precedent to leadership. 

Based on a seeming mismatch between the constructs highlighted in these theories and 
the behaviors described by higher education leaders, the review of leadership literature was 
expanded to include inclusive/collaborative (or "women's" ways; e.g., Astin & Leland, 1991; 
Helgesen, 1990, 1995), authentic (e.g., Henderson & Hoy, 1982), and team/group (e.g., Bradford 
& Cohen, 1998; Katzenbach & Smith, 1993) leadership theories. These theories provide better- 
fitting frameworks that explain the work of leaders in distributive, shared-governance 
organizations such as higher education. 

Personality theories were also tapped to explain the type and range of behaviors that fall 
within the higher education leadership framework. These theories include Thurstone’s (1934) 
“Big-5" personality theory and Jung’s (1971) psychological type preferences model. The 
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dichotomous distinctions in attributes and behaviors disclosed by the email survey also paralleled 
the distinctions made by traditional and behavioral approaches to personality assessment. The 
gist of the traditional approach suggests that human behavior is motivated by characteristics that 
individuals “have.” Behavioral approaches include what persons “do” in various situations to 
explain their behavior (Goldfried & Kent, 1972). 

Some leadership scholars have employed both traditional and behavioral approaches in 
explicating the concept of “authenticity” of actions (Etzioni, 1968; Henderson & Hoy, 1982). 
They define authenticity as congruence or alignment of a leader’s acts and beliefs; in effect, 
“matching the leader’s words with the leader’s actions” (Henderson & Hoy, 1982, p. 4). This 
notion of action and beliefs seemed to be reflected in the distinctions noted by higher education 
leaders who responded to the exploratory email survey between what they do and what they 
possess. 

Given these similarities, it was decided to further examine the construct of higher 
education leadership fi'om discrete innate (possessed) and extrinsic (directed toward others) 
points of view. Those behaviors displayed toward or with others (relational and visionary), 
were called “demonstrated resources.” The irmate attributes (humanistic and competence 
beliefs) were termed “core resources.” 

Conceptual Framework 

The systematic scrutiny of the literature on higher education and leadership and the 
preliminary email data described the first stage of developing higher education leadership’s 
content domain (Reise, Waller & Comrey, 2000). The following five dimensions emerged and 
were hypothesized to best define aspects of higher education leadership; they answer the 
question “What are the descriptors or components of higher education leadership?” 
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1 . Integral : This dimension captures the practices and behaviors that are necessary to 
enhance the organizational relationships in the administration of a shared governance 
system. Its indicators are inclusion, interdependence, and shared authority. 

2. Relational : This dimension captures the practices and behaviors that evidence the leader’s 
relationships on a personal level with members of the IHE. The indicators of this 
dimension are mentoring, inspiration, caring/sensitivity, and interpersonal skills. 

3. Credibility : This dimension includes values-based behaviors that exemplify leaders’ 
credibility. Its indicators are accountability, clarity of values, and confidence. 

4. Competence : This dimension defines the work ethic of leaders. The indicators of this 
dimension are distributed wisdom, hard work, and balance. 

5. Direction/Guidance : This dimension exemplifies leaders’ behaviors that direct the course 
of the IHE and its members as they confront internal and external challenges. Indicators 
of this dimension describe leaders who are visionary, change agents, and who challenge 
the status quo. 



The dimensions were then compared against five contemporary leadership assessment 
instruments to determine whether these same dimensions existed in any of them. Instruments 
that are commonly used in leadership assessment were chosen for this comparison— the Campbell 
Leadership Index (CLI; Campbell, 1998), the Leadership Practices Inventory (LPI; Kouzes & 
Posner, 1997), Leadership Skills Inventory (LSI; Anderson, 1999), Multifactor Leadership 
Questionnaire, Form 5x (MLQ; Bass & Avolio, 1995), The Leadership Profile (TLP; Rosenbach, 
Sashkin & Harburg, 1996). Criteria for their selection included (a) measurement of leadership 
attributes, practices, skills; (b) use of a multirater (360-degree) instrument, allowing for self- and 
others’ assessment; (c) prior testing on higher education populations; (d) available psychometric 
information. 

These instruments did not exhibit all of the higher education leadership dimensions 
derived to this point, justifying the development of a new instrument. 

With the content domain of higher education leadership established, ten hypotheses (two 
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hypotheses per dimension— one for its evinced [demonstrated] aspect, and the other for its innate 
[possessed] aspect) about the factorability of these dimensions were tested. Furthermore, it was 
hypothesized that the descriptors, or indicators, of these five dimensions, much like those that 
emerged in the critical behaviors email study earlier described, could be examined from 
dichotomous perspectives— behaviors that are evinced (“demonstrated resources”) and those that 
are possessed (“core resources”). This dimensional dichotomy is theorized to explain the 
authenticity of the higher education leader, answering the question “Does the higher education 
leader both possess and practice each descriptor of higher education leadership?” 

Methods/Data Sources 

Development of the HELI 

Thus, the dimensions of the latent construct of higher education leadership were 
identified, and their content and the relationships between their indicators were “mapped” 
according to the construct’s “nomological net,” or a “map” of the content and relationships of 
this construct (Cronbach & Meehl, 1955; Smith & Glass, 1987). The Higher Education 
Leadership Instrument (HELI) was home of the difficult process of “bootstrapping” the five- 
dimensional theory of higher education leadership with a measure (Nunnally & Bernstein, 1994). 

The next phase involved developing the items that represent these dimensions (Hinkin, 
1995; Schwab, 1980). Construction of the HELI items focused on how they (a) were phrased, 

(b) related to the domain, and (c) pointed the respondent to what was demanded (Nunnally & 
Bernstein, 1994). Care was taken so that items would be understood in a literal and pragmatic 
sense; simplicity, familiarity, and unambiguousness were imperative in developing the items for 
the instrument (Schwarz, 1997). 

Initially, the item pool was substantially over-inclusive with items that are representative 
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of both theoretically guided or empirically verified aspects of the construct; variations of these 
aspects were also included (Reise et al., 2000). Once the pool was established, each item was 
evaluated and rated for inclusion in the HELL A total of 64 items were placed into the first 
version of the instrument. Each dimension contains three to four measures of each indicator; 
one-half of the 64 items reflect the core resources of each dimension and the other half reflect its 
demonstrated resources. 

The pilot version of the HELI contained three sections. The first section contained the 64 
higher education leadership items. The second group consisted of demographic items (age, 
gender, position, number of years in position, and institution type). The third section contained 
open-ended questions to elicit the respondents’ comments and opinions about survey content and 
delivery and their suggestions for improving and refining the instrument. 

Where the first two stages of instrument development provide for establishing validity 
and reliability of the HELI, the third stage involves further construct validation, or “elaborating 
the nomological net” (Cronbach & Meehl, 1955). This includes tests of criterion-related validity 
(assessing two groups for differences on the measure), and discriminant and convergent validity 
(by using, for example, the multitrait-multimethod matrix developed by Campbell & Fiske 
(1959)). However, given this study’s pilot status, only principal components analyses (PC A) to 
test the theorized five-dimension structure of (a) demonstrated resource variables and (b) core 
resource variables of the HELI and a regression analysis to examine the relationship between the 
demonstrated and core resource factors were conducted; subsequent validation studies will 
follow. 

Administration of HELI 

Sample . The sample in this study (n=452) comprised deans randomly drawn from six 
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different types of colleges (agriculture, arts & sciences, business, education, engineering, and 
nursing) in IHEs assigned Carnegie classifications of Doctoral/Research University—Extensive 
and "Intensive in the Higher Education Directory (2001 ed., Rodenhouse, 2000). The 
determination was made to sample only deans in this pilot study because they comprised the 
largest group of higher education administrators in the Higher Education Directory from which 
to draw a random sample and still have sufficient numbers remaining that could be surveyed 
once the instrument was tested and revised. 

Web Survey . The HELI was developed for administration over the World Wide Web. It 
was anticipated that administration through this medium would yield a higher and faster response 
rate. Dillman’s (2000) survey design principles were observed in constructing the HELI. A 
server domain was acquired and the website Iwww.the heli.com) was established through a 
third-party internet service provider. Protocols were established for secure access to the website, 
instantaneous data capture to a downloadable spreadsheet, and automatic prompting for 
incomplete or unacceptable responses. 

Results 

A total of 232 responses were received, for a 5 1% response rate. The scores were 
captured in two datasets— 32 demonstrated resource items (HELI-D) and 32 core resource items 
(HELI-C)— for purposes of testing. 

Internal consistency of the instrument was high (alpha ranged from .74 to .91). Principal 
components analyses (PCA) tested the 10 hypotheses related to whether the indicators of the 
demonstrated and core resources would load on the five respective dimensions. In all PCAs, a 
general leadership factor emerged that accounted for the lion’s share of the test variance, along 
with one smaller (“integral”) factor, and other bipolar, ambiguous factors. While these findings 
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did not support the 10 research hypotheses, they indicated the instrument tapped the higher 
education leadership construct. 

Regression analysis revealed a strong association between the demonstrated and core 
behaviors, confirming all five research hypotheses. The high, positive (.722 to .854) correlation 
coefficients for each dichotomous dimension reflected strong agreement by deans that they align 
their behaviors with their beliefs; in essence, they “practice what they preach” (Henderson, 

1995, 1998; Henderson & Hoy, 1982). The strong associations indicated that the deans believed 
in what they do in the course of their own leadership, no matter what leadership task or 
responsibility is given to them. Leaders are developed to behave according to the paradigm that 
most adequately suits the work environment, but the way these behaviors are demonstrated 
emanates from each person differently. In the case of the HELI deans, it appears that their 
behaviors were consistent with their personal beliefs. 

Though the results of this pilot study indicate a need to revise many of the HELI items to 
improve upon the instrument’s clarity and definition and/or to redefine the dimensions of the 
higher education leadership construct, it is clear that the general findings support the notion that 
higher education leadership, at least from the standpoint of the deans studied herein, does indeed 
involve facets of the five theorized dimensions. That one general factor emerged in each dataset 
indicates the higher education leadership construct was tapped. That the second (albeit, smaller) 
factor derived in each dataset comprised the “integral” dimension confirms the existence of the 
concepts of inclusion, interdependence, networking, and shared authority in these deans from the 
perspectives that they practice them and believe in their relevance. 

Discussion 

In this study, a systematic approach was taken to identify the attributes and behaviors of 
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higher education leaders and to translate them, based on relationships to the requirements of 
higher education administration, into an instrument to assess leadership for higher education. It 
is not an instrument to test a higher education leader’s effectiveness but, rather, to determine 
what leaders perceive as attributes or behaviors necessary for effective leadership. Continued 
validation studies with the instrument will provide a bird’s-eye view of respondents’ “take” on 
these dimensions. For example, we may see that respondents are unable to distinguish the items 
in their own minds and will continue to regard some or all of these dimensions as inextricably 
connected. Credibility and competence may be subsumed in each dimension of higher education 
leadership and therefore not be considered by respondents as attributes that can stand alone. A 
leader who is able to guide the direction of the organization in the face of adversity must be 
perceived as competent by those affected in the organization. Maintaining a working 
relationship with critical outsiders while keeping the mission of the organization at the forefront 
is facilitated by the leader’s credibility. The general factor that emerged in each of the HELI 
PCAs may be an indication of the inextricability of components; that is, that they are seen as 
necessarily working in concert. Therefore, it seems likely that restructuring the dimensions to 
account for this overlap may occur, once the HELI item content and representativeness test 
consistently. This restructuring may involve combining and/or relabeling dimensions to reflect 
the combination of constructs, revisiting the dimensions that describe leaders’ behaviors, or 
defining context-specific behaviors. 

But prudence and good practice dictate that other, more diverse methods be employed in 
explicating the dimensionality of higher education leadership. Items earmarked for revision 
should be pretested in cognitive interviews to determine whether respondents comprehend them 
as they were intended (Dillman, 2000). Another informal email study might be employed to ask 
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more questions about perceptions of leadership from various administrators, and this exercise 
may help to further delineate the dimensions of higher education leadership. Case studies or 
focus groups could also be conducted to assess practicing administrators’ perceptions of 
leadership in higher education. Narratives generated therefrom will no doubt provide detailed 
insight to the relationships that the HELI tried to explicate. An experimental design could be 
implemented to test differences of groups of administrators on different variables. This addition 
of richer data will continue to elaborate the construct’s nomological net, thereby allowing for 
further testing and revision of the HELI, and/or redefinition of the construct of higher education 
leadership. 

Though construct validation continues, the information derived thus far points us toward 
consistent, relevant, and meaningful methods for developing persoimel in higher education 
programs, no matter from what field or discipline they emerged. Some ideas for methods to 
employ in developing leaders along the HELI dimensions are discussed below. 

Tied directly to the “integral” dimension of higher education leadership is the ability to 
function in an organization that includes diverse authority, perspectives, and disciplines. The 
work involves balancing the demands and needs of the professoriate, regent, administrative, and 
student factions. In this regard, leadership development programs may encourage collaborative 
learning and cooperation and developing a relational approach to learning (Belenky, Clinchy, 
Goldberger & Tarule, 1986; Murrell & Davis, 1991). Courses that emphasize team building and 
teamwork will facilitate building coimections between EHEs and agencies or communities 
outside of it (Hodgkinson, 1985; Katzenbach & Smith, 1993). In order to encourage an 
interactive and interdependent atmosphere in administration, programs might emphasize the 
beneficial effect of combining different leadership perspectives with different personal 
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definitions (Kezar, 2002). 

From the “relational” dimension of higher education leadership, we see that a leader’s 
interpersonal relationships with others is not just about caring and being able to inspire. It is also 
about resolving conflict, dealing with difficult people, and mentoring (Townsend & Bassoppo- 
Moyo, 1996). Leadership development programs should encourage a “mentoring community” 
that instills responsibility in, and commitment to, one another (Murrell & Davis, 1991). They 
might also focus on human resource management and personnel administration in both education 
and other disciplines, to emphasize the distinctions and similarities in communication and 
business practices and skills. Studies in group dynamics and decision-making no doubt will 
enhance relationships in practice (Katzenbach & Smith, 1993). 

“Credibility,” the third dimension of higher education leadership, describes the values- 
based behaviors of leaders. Ethics courses should target development of a sense of integrity and 
a strong moral commitment (Murrell & Davis, 1991). Outside of courses, higher education 
leaders might be counseled to learn self-reflection and awareness of identity (Kezar, 2002). 

The dimension of “competence” is the area that appears to have greatest attention in 
higher education leadership literature. Developing high-level organization skills, and proficiency 
in budget, finance and technology, are imperative (Murrell & Davis, 1991; Townsend & 
Bassoppo-Moyo, 1996). Understanding the physical workings of the administration is facilitated 
by courses in planning, governance, and organizational policy (McDade, 1987). Programs must 
also encourage understanding leaders’ own capacity for personal growth, the life cycle stages in 
academic and administrative life, and the importance of maintaining balance in their lives (Astin 
& Leland, 1991; Helgesen, 1995; Murrell & Davis, 1991). 

Finally, the “direction/guidance” dimension informs programs on the development of 
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leaders’ abilities to move forward in a climate of conflict and change. Focus may be upon 
organizational change and contact with external educational and political agencies to encourage 
leaders’ understanding of the greater social context in which higher education exists. Implicit in 
these programs should be opportunities to engage in critical thinking, problem solving, and 
tolerance for ambiguity and paradox (Murrell & Davis, 1991). In this way, leaders can better 
appreciate the multiple interpretations of institutional leadership in order to shape the vision as a 
collective one. 

Conclusion 

The HELI, though still in its developmental stages, provides one means by which 
continued exploration of higher education leadership may be conducted. But it is only one tool 
that must be used with many, just as a CT scanner provides one means of diagnosing a medical 
condition. Like a single image produced in a CT scan, this study represents one slice of the 
whole picture of higher education leadership. With further study, it is hoped more “slices” will 
emerge to complete the picture. This, in turn, will provide valuable insight to programs for 
developing leaders so they will more capably and confidently navigate the leadership quagmire 
in higher education today. 
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